Learning to Scale Payments in Crowdsourcing with PropeRBoost
نویسندگان
چکیده
Motivating workers to provide significant effort has been recognized as an important issue in crowdsourcing. It is important not only to compensate worker effort, but also to discourage low-quality workers from participating. Several proper incentive schemes have been proposed for this purpose; they are either based on gold tasks or on peer consistency in individual tasks. As the rewards cannot become negative, these schemes have difficulty in achieving zero expected reward for random answers. We describe a novel boosting scheme, ProperRBoost, that improves the efficiency of existing incentive schemes by making a better separation between incentives for high and low quality work, and effectively discourages random answers by assigning them near minimal average rewards. We show the actual performance of the boosting scheme through simulations of various worker strategies.
منابع مشابه
Perform Three Data Mining Tasks with Crowdsourcing Process
For data mining studies, because of the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. Uncertainty about the performance of virtual user...
متن کاملObtaining High-Quality Label by Distinguishing between Easy and Hard Items in Crowdsourcing
Crowdsourcing systems make it possible to hire voluntary workers to label large-scale data by offering them small monetary payments. Usually, the taskmaster requires to collect high-quality labels, while the quality of labels obtained from the crowd may not satisfy this requirement. In this paper, we study the problem of obtaining high-quality labels from the crowd and present an approach of le...
متن کاملA Deterministic MAB Mechanism for Crowdsourcing with Logarithmic Regret and Immediate Payments
We consider a general crowdsourcing setting with strategic workers whose qualities are unknown and design a multiarmed bandit (MAB) mechanism, CrowdUCB, which is deterministic, regret minimizing, and offers immediate payments to the workers. The problem involves sequentially selecting workers to process tasks in order to maximize the social welfare while learning the qualities of the strategic ...
متن کاملWorkshops Held at the First AAAI Conference on Human Computation and Crowdsourcing: A Report
Crowdsourcing (HCOMP-2013) was held November 6– 9, 2013, in Palm Springs, California. Three workshops took place on Saturday, November 9: Crowdsourcing at Scale, Disco: Human and Machine Learning in Games, and Scaling Speech, Language Understanding, and Dialogue through Crowdsourcing. The goal of the Crowdsourcing at Scale workshop was to identify the biggest challenges in very large-scale crow...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016